The present disclosure relates generally to user interfaces, and in particular to a user interface that enables a user to calibrate the position of a three dimensional model with a real-world environment represented by that model.
Some applications are being developed for mobile devices which allow users to view an “augmented reality” or “enhanced reality” view of their real, physical environment. Some mobile devices, including cell phones and so-called smart phones, come equipped with cameras and graphical display screens. A camera may be mounted on the side of the device that is opposite of the side on which the device's display is placed. Enhanced reality applications typically capture the contents currently within the view of the camera in real time and present those contents on the display. Thus, the mobile device's user can hold up the mobile device in front of his field of vision in order to use the device's graphical display as a sort of camera view finder. The device's display shows the user everything that the camera can see. However, enhanced reality applications go further than simply presenting a view of the real world to the device's user. Enhanced reality applications seek to register a projected model of the world with live video of the world. Enhanced reality applications are frequently capable of overlaying real-world view with information about the people and objects contained within that view. For example, an enhanced reality application might overlay a particular building within a view with the name of the corporation whose office the building houses. For another example an enhanced reality application might overlay a particular street within a view with its name. For yet another example, more sophisticated applications might even seek to present the names of people within a view within virtual text bubbles floating above those people's faces.
In order to overlay information over a particular object within the real world view in an accurate manner, the enhanced reality application must have some way of determining which of the many pixels within the view correspond to that particular object. The real world view might contain various pixels corresponding to many different objects in various different locations. If the overlaid information is placed within the wrong spot within the view, then the user of the enhanced reality application may become confused or misinformed. Therefore, enhanced reality applications may refer to a virtual model of the real world. This virtual model might include, for each known real-world object, a set of three dimensional coordinates that represent points defining the contours of the surface of that object. For example, the set of coordinates for a building would probably include at least coordinates for the points at the corners of the building. If the coordinates corresponding to a particular object are connected with lines, a wire-frame model of that particular object may be produced.
Possessing a particular object's set of coordinates, or its wire-frame model, helps very little unless those coordinates are accurate. An object model database may include, for various real-world objects, coordinates that are meant to be meaningful within a coordinate system that is used by a global positioning system (GPS). When the user of an enhanced realty application uses his application at a particular place, the device on which that application executes may utilize its sensors—typically including a compass, a gyro, and a GPS—to estimate device attitude and displacement in reference to a given frame, and the GPS coordinates at which the device is positioned. The application may use this estimated location, attitude, and displacement information obtained automatically by the sensors in order to attempt to place the user accurately within a virtual model. If the user's position and orientation were estimated with perfect accuracy, and if the model to be portrayed were perfect, then the wire frame model of objects within the user's vicinity could be aligned perfectly accurately with the pixels that correspond to objects within the device's view. Under such circumstances, the enhanced information could be overlaid at precisely the correct positions within the enhanced reality view. Unfortunately, because GPS is a satellite-based technology, its positional estimates are not fine-grained, and often can only be obtained at a relatively coarse granularity, so that the user's actual coordinates within the wire frame model might be significantly offset from the coordinates estimated using the mobile device's GPS. Furthermore, inaccuracies in alignment between a wire frame model and the pixels within the device's view can result from errors in orientation, attitude, and displacement due to inaccuracies in compass, gyro, and tilt measurements. Consequently, even the best automatically ascertained coordinate and orientation estimates for the user can cause the enhanced reality application to align the wire frame model with the real world view imprecisely, producing errors in the manner in which enhanced information is overlaid in the enhanced view.